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Abstract 

Two philosophical applications of the concept of program-size com- 
plexity are discussed. First, we consider the light program-size com- 
plexity sheds on whether mathematics is invented or discovered, i.e., 
is empirical or is a priori. Second, we propose that the notion of al- 
gorithmic independence sheds light on the question of being and how 
the world of our experience can be partitioned into separate entities. 

1. Introduction. Why is program size of philo- 
sophical interest? 

The cover of the January 2003 issue of La Recherche asks this dramatic 
question: 

Dieu est-il un ordinateur? [Is God a computer?] 

The long cover story [1] is a reaction to Stephen Wolfram's controversial book 
A New Kind of Science [2]. The first half of the article points out Wolfram's 
predecessors, and the second half criticizes Wolfram. 

The second half of the article begins (p. 38) with these words: 

II [Wolfram] n'avance aucune raison serieuse de penser que les 
complexites de la nature puissent etre generees par des regies 
enongables sous forme de programmes informatiques simples. 
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The reason for thinking that a simple program might describe the world is, 
basically, just Plato's postulate that the universe is rationally comprehensible 
(Timaeus). A sharper statement of this principle is in Leibniz's Discours de 
metaphysique [3], section VI. Here is Leibniz's original French (1686): 

Mais Dieu a choisi celuy qui est le plus parfait, c'est a dire celuy 
qui est en meme temps le plus simple en hypotheses et le plus riche 
en phenomenes, comme pourroit estre une ligne de Geometrie 
dont la construction seroit aisee et les proprietes et effects seroient 
fort admirables et d'une grande etendue. 

For an English translation of this, see [4] . 

And Hermann Weyl [5] discovered that in Discours de metaphysique Leib- 
niz also states that a physical law has no explicative power if it is as compli- 
cated as the body of data it was invented to explain. 1 

This is where algorithmic information theory (AIT) comes in. AIT posits 
that a theory that explains X is a computer program for calculating X, that 
therefore must be smaller, much smaller, than the size in bits of the data 
X that it explains. AIT makes a decisive contribution to philosophy by 
providing a mathematical theory of complexity. AIT defines the complexity 
or algorithmic information content of X to be the size in bits H{X) of the 
smallest computer program for calculating X. H(X) is also the complexity 
of the most elegant (the simplest) theory for X. 

In this article we discuss some other philosophical applications of AIT. 

For those with absolutely no background in philosophy, let me recommend 
two excellent introductions, Magee [6] and Brown [7]. For introductions to 
AIT, see Chaitin [8, 9]. For another discussion of the philosophical implica- 
tions of AIT, see Chaitin [10]. 

2. Is mathematics empirical or is it a priori? 
2.1. Einstein: math is empirical 

Einstein was a physicist and he believed that math is invented, not discov- 
ered. His sharpest statement on this is his declaration that "the series of 

1 See the Leibniz quote in Section 2.3 below. 
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integers is obviously an invention of the human mind, a self-created tool 
which simplifies the ordering of certain sensory experiences." 
Here is more of the context: 

In the evolution of philosophic thought through the centuries the 
following question has played a major role: What knowledge is 
pure thought able to supply independently of sense perception? 
Is there any such knowledge?. . . I am convinced that. . . the con- 
cepts which arise in our thought and in our linguistic expressions 
are all. . . the free creations of thought which can not inductively 
be gained from sense-experiences. . . Thus, for example, the 
series of integers is obviously an invention of the human 
mind, a self-created tool which simplifies the ordering of 
certain sensory experiences. 2 

The source is Einstein's essay "Remarks on Bertrand Russell's theory of 
knowledge." It was published in 1944 in the volume [11] on The Philosophy 
of Bertrand Russell edited by Paul Arthur Schilpp, and it was reprinted in 
1954 in Einstein's Ideas and Opinions [12]. 

And in his Autobiographical Notes [13] Einstein repeats the main point 
of his Bertrand Russell essay, in a paragraph on Hume and Kant in which he 
states that "all concepts, even those closest to experience, are from the point 
of view of logic freely chosen posits." Here is the bulk of this paragraph: 

Hume saw clearly that certain concepts, as for example that of 
causality, cannot be deduced from the material of experience by 
logical methods. Kant, thoroughly convinced of the indispens- 
ability of certain concepts, took them. . . to be the necessary 
premises of any kind of thinking and distinguished them from 
concepts of empirical origin. I am convinced, however, that this 
distinction is erroneous or, at any rate, that it does not do justice 
to the problem in a natural way. All concepts, even those 
closest to experience, are from the point of view of logic 
freely chosen posits. . . 

2 [The boldface emphasis in this and future quotations is mine, not the author's.] 
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2.2. Godel: math is a priori 



On the other hand, Godel was a Platonist and believed that math is a pri- 
ori. He makes his position blindingly clear in the introduction to an unpub- 
lished lecture Godel *1961 /?, "The modern development of the foundations 
of mathematics in the light of philosophy," Collected Works [14], vol. 3: 3 

I would like to attempt here to describe, in terms of philosophical 
concepts, the development of foundational research in mathemat- 
ics. .. , and to fit it into a general schema of possible philosophical 
world- views [Weltanschauungen] ... I believe that the most fruit- 
ful principle for gaining an overall view of the possible world- views 
will be to divide them up according to the degree and the manner 
of their affinity to or, respectively, turning away from metaphysics 
(or religion). In this way we immediately obtain a division into 
two groups: skepticism, materialism and positivism stand on one 
side, spiritualism, idealism and theology on the other. . . Thus 
one would, for example, say that apriorism belongs in principle 
on the right and empiricism on the left side. . . Now it is a familiar 
fact, even a platitude, that the development of philosophy since 
the Renaissance has by and large gone from right to left. . . It 
would truly be a miracle if this (I would like to say rabid) devel- 
opment had not also begun to make itself felt in the conception of 
mathematics. Actually, mathematics, by its nature as an a 
priori science, always has, in and of itself, an inclination toward 
the right, and, for this reason, has long withstood the spirit 
of the time [Zeitgeist] that has ruled since the Renaissance; i.e., 
the empiricist theory of mathematics, such as the one set forth 
by Mill, did not find much support. . . Finally, however, around 
the turn of the century, its hour struck: in particular, it was the 
antinomies of set theory, contradictions that allegedly appeared 
within mathematics, whose significance was exaggerated by skep- 
tics and empiricists and which were employed as a pretext for the 
leftward upheaval. . . 

3 The numbering scheme used in Godel's Collected Works begins with an * for unpub- 
lished papers, followed by the year of publication, or the first/last year that Godel worked 
on an unpublished paper. 
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Nevertheless, the Platonist Godel makes some remarkably strong state- 
ments in favor of adding to mathematics axioms which are not self-evident 
and which are only justified pragmatically. What arguments does he present 
in support of these heretical views? 

First let's take a look at his discussion of whether Cantor's continuum 
hypothesis could be established using a new axiom [Godel 1947, "What is 
Cantor's continuum problem?", Collected Works, vol. 2]: 

. . . even disregarding the intrinsic necessity of some new 
axiom, and even in case it has no intrinsic necessity at all, a 
probable decision about its truth is possible also in an- 
other way, namely, inductively by studying its "success." 
Success here means fruitfulness in consequences, in particular in 
"verifiable" consequences, i.e., consequences demonstrable with- 
out the new axiom, whose proofs with the help of the new ax- 
iom, however, are considerably simpler and easier to discover, and 
make it possible to contract into one proof many different proofs. 
The axioms for the system of real numbers, rejected by intu- 
itionists, have in this sense been verified to some extent, owing 
to the fact that analytical number theory frequently allows one 
to prove number-theoretical theorems which, in a more cumber- 
some way, can subsequently be verified by elementary methods. 
A much higher degree of verification than that, however, is con- 
ceivable. There might exist axioms so abundant in their ver- 
ifiable consequences, shedding so much light upon a whole field, 
and yielding such powerful methods for solving problems (and 
even solving them constructively, as far as that is possible) that, 
no matter whether or not they are intrinsically necessary, they 
would have to be accepted at least in the same sense as 
any well-established physical theory. 

Later in the same paper Godel restates this: 

It was pointed out earlier. . . that, besides mathematical in- 
tuition, there exists another (though only probable) crite- 
rion of the truth of mathematical axioms, namely their 
fruitfulness in mathematics and, one may add, possibly also in 
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physics. . . The simplest case of an application of the criterion un- 
der discussion arises when some. . . axiom has number-theoretical 
consequences verifiable by computation up to any given integer. 

And here is an excerpt from Godel's contribution [Godel 1944, "Russell's 
mathematical logic," Collected Works, vol. 2] to the same Bertrand Russell 
festschrift volume [11] that was quoted above: 

The analogy between mathematics and a natural science is en- 
larged upon by Russell also in another respect. . . axioms need 
not be evident in themselves, but rather their justifica- 
tion lies (exactly as in physics) in the fact that they make 
it possible for these "sense perceptions" to be deduced. . . 
I think that. . . this view has been largely justified by subsequent 
developments, and it is to be expected that it will be still more 
so in the future. It has turned out that the solution of certain 
arithmetical problems requires the use of assumptions essentially 
transcending arithmetic. . . Furthermore it seems likely that for 
deciding certain questions of abstract set theory and even for cer- 
tain related questions of the theory of real numbers new axioms 
based on some hitherto unknown idea will be necessary. Perhaps 
also the apparently insurmountable difficulties which some other 
mathematical problems have been presenting for many years are 
due to the fact that the necessary axioms have not yet been found. 
Of course, under these circumstances mathematics may lose a 
good deal of its "absolute certainty;" but, under the influence of 
the modern criticism of the foundations, this has already hap- 
pened to a large extent. . . 

Finally, take a look at this excerpt from Godel *1951, "Some basic theo- 
rems on the foundations," Collected Works, vol. 3, an unpublished essay by 
Godel: 

I wish to point out that one may conjecture the truth of a uni- 
versal proposition (for example, that I shall be able to verify a 
certain property for any integer given to me) and at the same 
time conjecture that no general proof for this fact exists. It is 
easy to imagine situations in which both these conjectures would 
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be very well founded. For the first half of it, this would, for exam- 
ple, be the case if the proposition in question were some equation 
F(n) = G(n) of two number-theoretical functions which could be 
verified up to very great numbers n. A Moreover, exactly as in the 
natural sciences, this inductio per enumerationem simplicem is by 
no means the only inductive method conceivable in mathemat- 
ics. I admit that every mathematician has an inborn abhorrence 
to giving more than heuristic significance to such inductive argu- 
ments. I think, however, that this is due to the very prejudice that 
mathematical objects somehow have no real existence. If math- 
ematics describes an objective world just like physics, 
there is no reason why inductive methods should not be 
applied in mathematics just the same as in physics. The 
fact is that in mathematics we still have the same attitude today 
that in former times one had toward all science, namely, we try to 
derive everything by cogent proofs from the definitions (that is, 
in ontological terminology, from the essences of things). Perhaps 
this method, if it claims monopoly, is as wrong in mathematics 
as it was in physics. 

So Godel the Platonist has nevertheless managed to arrive, at least par- 
tially, at what I would characterize, following Tymoczko [16], as a pseudo- 
empirical or a quasi-empirical position! 

2.3. AIT: math is quasi-empirical 

What does algorithmic information theory have to contribute to this dis- 
cussion? Well, I believe that AIT also supports a quasi-empirical view of 
mathematics. And I believe that it provides further justification for Godel's 
belief that we should be willing to add new axioms. 
Why do I say this? 

As I have argued on many occasions, AIT, by measuring the complexity 
(algorithmic information content) of axioms and showing that Godel incom- 
pleteness is natural and ubiquitous, deepens the arguments that forced Godel, 

4 Such a verification of an equality (not an inequality) between two number-theoretical 
functions of not too complicated or artificial structure would certainly give a great 
probability to their complete equality, although its numerical value could not be estimated 
in the present state of science. However, it is easy to give examples of general propositions 
about integers where the probability can be estimated even now. . . 
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in spite of himself, in spite of his deepest instincts about the nature of math- 
ematics, to believe in inductive mathematics. And if one considers the use of 
induction rather than deduction to establish mathematical facts, some kind 
of notion of complexity must necessarily be involved. For as Leibniz stated in 
1686, a theory is only convincing to the extent that it is substantially simpler 
than the facts it attempts to explain: 

. . . non seulement rien n'arrive dans le monde, qui soit absolument 
irregulier, mais on ne sgauroit memes rien feindre de tel. Car 
supposons par exemple que quelcun fasse quantite de points sur le 
papier a tout hazard, comme font ceux qui exercent l'art ridicule 
de la Geomance, je dis qu'il est possible de trouver une ligne 
geometrique dont la motion soit constante et uniforme suivant 
une certaine regie, en sorte que cette ligne passe par tous ces 
points. . . Mais quand une regie est fort composee, ce qui 
luy est conforme, passe pour irregulier. Ainsi on peut dire 
que de quelque maniere que Dieu auroit cree le monde, il auroit 
tousjours este regulier et dans un certain ordre general. Mais 
Dieu a choisi celuy qui est le plus parfait, c'est a dire celuy qui 
est en meme temps le plus simple en hypotheses et le plus 
riche en phenomenes. . . [Discours de metaphysique, VI] 

In fact Godel himself, in considering inductive rather than deductive 
mathematical proofs, began to make some tentative initial attempts to for- 
mulate and utilize notions of complexity. (I'll tell you more about this in a 
moment.) And it is here that AIT makes its decisive contribution to philos- 
ophy, by providing a highly-developed and elegant mathematical theory of 
complexity. How does AIT do this? It does this by considering the size of 
the smallest computer program required to calculate a given object X, which 
may also be considered to be the most elegant theory that explains X. 

Where does Godel begin to think about complexity? He does so in two 
footnotes in vol. 3 of his Collected Works. The first of these is a footnote 
to Godel *1951. This footnote begins "Such a verification..." and it was 
reproduced, in part, in Section 2.2 above. And here is the relevant portion 
of the second, the more interesting, of these two footnotes: 

. . . Moreover, if every number-theoretical question of Goldbach 
type. . . is decidable by a mathematical proof, there must exist an 
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infinite set of independent evident axioms, i.e., a set m of evident 
axioms which are not derivable from any finite set of axioms (no 
matter whether or not the latter axioms belong to m and whether 
or not they are evident). Even if solutions are desired only for all 
those problems of Goldbach type which are simple enough to be 
formulated in a few pages, there must exist a great number of 
evident axioms or evident axioms of great complication, in 
contradistinction to the few simple axioms upon which 
all of present day mathematics is built. (It can be proved 
that, in order to solve all problems of Goldbach type of a certain 
degree of complication k, one needs a system of axioms whose 
degree of complication, up to a minor correction, is > A;.) 5 

This is taken from Godel *1953/9— III, one of the versions of his unfinished 
paper "Is mathematics syntax of language?" that was intended for, but was 
finally not included, in Schilpp's Carnap festschrift in the same series as the 
Bertrand Russell festschrift [11]. 

Unfortunately these tantalizing glimpses are, as far as I'm aware, all that 
we know about Godel's thoughts on complexity. Perhaps volumes 4 and 
5, the two final volumes of Godel's Collected Works, which contain Godel's 
correspondence with other mathematicians, and which will soon be available, 
will shed further light on this. 

Now let me turn to a completely different — but I believe equally 
fundamental — application of AIT. 

3. How can we partition the world into distinct 
entities? 

For many years I have asked myself, "What is a living being? How can we 
define this mathematically?!" I still don't know the answer! But at least 

5 [This is reminiscent of the theorem in AIT that pk = (the program of size < k bits 
that takes longest to halt) is the simplest possible "axiom" from which one can solve the 
halting problem for all programs of size < k. Furthermore, pkS size and complexity both 
differ from k by at most a fixed number of bits: \pk\ — k + 0(1) and H(pk) = k + 0(1). 

Actually, in order to solve the halting problem for all programs of size < k, in addition 
to pk one needs to know k — \pk\, which is how much pk's size differs from k. This fixed 
amount of additional information is required in order to be able to determine k from p k .] 
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I think I now know how to come to grips with the more general notion of 
"entity" or "being." In other words, how can we decompose our experience 
into parts? How can we partition the world into its components? By what 
right do we do this in spite of mystics who like Parmenides insist that the 
world must be perceived as an organic unity (is a single substance) and 
cannot be decomposed or analized into independent parts? 

I believe that the key to answering this fundamental question lies in AIT's 
concept of algorithmic independence. What is algorithmic independence? 
Two objects X and Y are said to be algorithmically independent if their 
complexity is (approximately) additive. In other words, X and Y are algo- 
rithmically independent if their information content decomposes additively, 
i.e., if their joint information content (the information content of X and Y) 
is approximately equal to the sum of their individual information contents: 

H(X,Y) « H(X) + H(Y). 

More precisely, the left-hand side is the size in bits of the smallest program 
that calculates the pair X, Y, and the right-hand side adds the size in bits 
of the smallest program that produces X to the size in bits of the smallest 
program that calculates Y. 

Contrariwise, if X and Y are not at all independent, then it is much 
better to compute them together than to compute them separately and 
H(X) + H(Y) will be much larger than H(X, Y). The worst case is X = Y. 
Then H(X) + H(Y) is twice as large as H(X, Y). 

I feel that this notion of algorithmic independence is the key to decom- 
posing the world into parts, parts the most interesting example of which are 
living beings, particularly human beings. For what enables me to partition 
the world in this way? The fact that thinking of the world as a sum of such 
parts does not complicate my description of the world substantially and at 
the same time enables me to use separate subroutines such as "my wife" and 
"my cat" in thinking about the world. That is why such an analysis of the 
world, such a decomposition, works. 

Whereas on the contrary "my left foot" and "my right hand" are not 
well thought of as independent components of the world but can best be 
understood as parts of me. A description of my right hand and its activities 
and history would not be substantially simpler than a description of me and 
my entire life history, since my right hand is a part of me whose actions 
express my intentions, and not its own independent desires. 
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Of course, these observations are just the beginning. A great deal more 
work is needed to develop this point of view. . . 

For a technical discussion of algorithmic independence and the associated 
notion of mutual algorithmic information defined as follows 

H(X : Y) = H(X) + H(Y) - H(X, Y), 

see my book Chaitin [17]. 

4. Conclusion and future prospects 

Let's return to our starting point, to the cover of the January 2003 issue of 
La Recherche. Is God a computer, as Wolfram and some others think, or is 
God, as Plato and Pythagoras affirm, a mathematician? 

And, an important part of this question, is the physical universe dis- 
crete, the way computers prefer, not continuous, the way it seems to be 
in classical Newtonian/Maxwellian physics? Speaking personally, I like the 
discrete, not the continuous. And my theory, AIT, deals with discrete, digital 
information, bits, not with continuous quantities. But the physical universe 
is of course free to do as it likes! 

Hopefully pure thought will not be called upon to resolve this. Indeed, I 
believe that it is incapable of doing so; Nature will have to tell us. Perhaps 
someday an experimentum cruets will provide a definitive answer. In fact, 
for a hundred years quantum physics has been pointing insistently in the 
direction of discreteness. 6 
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